Multicanonical distribution and the origin of power laws 

G. L. VasconceloJ*] and D. S. P. SalazaJ3 

Departamento de Fisica, Universidade Federal de Pernambuco, Brazil, and 
Unidade de Educagao a Distdncia e Tecnologia, 
Universidade Federal Rural de Pernambuco, Brazil. 

Abstract 

A multicanonical formalism is applied to the problem of statistical equilibrium in a complex 
system with a hierarchy of dynamical structures. At the small scales the system is in quasi- 
equilibrium and follows a Maxwell-Boltzmann distribution with a slowly fluctuating temperature. 
The probability distribution for the temperature is determined using Bayesian analysis and it is 
then used to average the Maxwell-Boltzmann distribution. The resulting energy distribution law 
is written in terms of generalized hypergeometric functions, which display powerdaw tails. 
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I. INTRODUCTION 



Power law distributions occur in widely diverse physical systems spanning an impressive 
range of length scales The appearance of heavy-tailed distributions is often traced to 
the presence of hierarchical structures in the system |2|, whose complex "interaction" may 
result in violation of the statistical independence of subsystems, thus leading to non-Gibbsian 
distributions [3]. Despite these insights and the many contributions to the problem [4J, it is 
fair to say that the physical mechanisms behind the emergence of power-law distributions 
are not yet well understood. 

Recently, we introduced js] a general formalism to describe statistical equilibrium of 
complex systems with multiple scales where the probability distribution of states displays 
power-law tails. In this formalism, the hierarchical structure embedding the system of inter- 
est is effectively modeled as a set of nested "internal heat reservoirs," where each "reservoir" 
is described by only one effective degree of freedom, namely, its "temperature." On the basis 
of a few physically reasonable assumptions, it was possible to show that for a large class of 
systems the equilibrium distribution can be written explicitly in terms of certain generalized 
hypergeometric functions, which exhibit power law tails. This family of generalized hy- 
pergeometric (GHG) distributions includes, as its first two members, the Boltzmann-Gibbs 
distribution and the Tsallis distribution (7|. Higher-order members of the GHG family of dis- 
tributions have been shown to describe remarkably well the statistics of velocity fluctuations 
in turbulence {5]. 

The GHG distribution represents a generalization of the canonical distribution for mul- 
tiscale systems and hence it is also called multicanonical. The main purpose of the present 
paper is to give an alternative derivation, based on Bayesian analysis, of the multicanonical 
distribution. Because the derivation of the multicanonical distribution given here tries to 
parallel (whenever possible) the usual treatment of the canonical distribution, we shall begin 
our presentation by briefly reviewing the derivation of the canonical distribution. 



II. THE CANONICAL DISTRIBUTION 



Consider a system in thermal equilibrium at some temperature To. We recall that in 
the canonical formalism the constant temperature constraint is enforced by embedding the 
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FIG. 1. A canonical system in thermal equilibrium at temperature To. Arrows indicate energy 
exchange between the subsystems. 
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We 



system is a much larger system (i.e., a heat bath) capable of giving it energy 
designate the system of interest by the label 2 and the larger system embedding it by the 
label 1, with the combined system formed by subsystems 1 and 2 being given the label 0; 
see Fig. 1. 

The energies of systems 1 and 2 will be denoted by E\ and Ef, respectively, where k 
and i represent the labels designating the possible states in each system. Let us denote by 
p (Ek t i) = po(E k + Ef) the probability of finding system in a state corresponding to energy 
E kji = E k + Ef. In view of the independence between systems 1 and 2, one can then write 

p (E k + E i )=p 1 (E k )p 2 (E i ), (1) 

where p\{E k ) and p2(Ei) are the probabilities of finding systems 1 and 2 in states with energy 
E k and E^, respectively. Taking the logarithm derivative of ([T]) with respect to E k yields 

d\np ( y E i + E k ) = dlnpi(E fc ) = _^ 
dE k dE k 

where Pi may be a function of E k but not of E{. On the other hand, it is clear that 

dhipo _ dlnpo _ d\np 2 (Ej) 

dE k ~ dEi dEi ' U 

Comparing (j2J) and ([3]), one then concludes that 

d\np 2 (E i ) 

dEi " (4) 
Since 0i does not depend on E iy the preceding equation can be readily integrated, yielding 

p 2 (Ei\fi\) = constant • exp(— (3iEi). (5) 
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Since the partition of system into subsystems 1 and 2 is entirely arbitrary, the quantity 
Pi must be the same for any partition one chooses. In other words, Pi is a characteristic of 
system only, that is, 



Pi = A>, 



(6) 



which implies that 



p 2 (E i \p ) = Aexp(-P Ei), 



(7) 



where A is a constant. 

It should be evident that the discussion above is completely symmetrical with respect to 
labels 1 and 2, so that system 2 (irrespective of its size) obeys the same distribution law as 
system 1. In then follows from ([1]) and ([7]) that the probability of finding any subsystem of 
system in a state with energy E is given by the Boltzmann-Gibbs (BG) distribution: 



is the partition function 

It is important to emphasize here that the key step in deriving ([8]) was the ability to par- 
tition the system into two independent subsystems of arbitrary sizes, so that each subsystem 
is described by the same distribution law. There are however many physical systems, where 
the relevant probability distributions depend on the scale at which the measurements are 
made. In such complex systems, the system cannot be partitioned into independent subsys- 
tems of arbitrary sizes, and one has to treat each dynamical scale separately, as discussed 
next. 

III. THE MULTICANONICAL DISTRIBUTION 

Here we consider a multiscale system of size L in thermal equilibrium at temperature To. 
We assume that the system possesses a hierarchy of dynamical structures of characteristic 
sizes £i = L/b 1 ^ 1 , with i — 1, 2, n, where b is a number greater than 1. (The specific value 




(8) 



where g(E)is the density of states and 




(9) 
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FIG. 2. A multicanonical system in thermal equilibrium at temperature Tq. 



of b is not relevant here.) We suppose furthermore that there is a wide separation of time 
scales within this hierarchy, with smaller structures having shorter characteristic times. Let 
us now consider a partition of our system into "nested" subsystems of sizes £j, as indicated 
in Fig. 2. We shall designate the subsystem of size £j by the label j. The "thermodynamic 
state" of each subsystem j will be characterized by only one parameter, namely, its inverse 
temperature /3j. As before, we designate the combined system consisting of all subsystems 
by the label 0. 

Let us now focus our attention on the subsystem n of size £ n . Since £ n is the smallest 
characteristic length scale in the system, it is clear that this subsystem can be arbitrarily 
divided into two independent subsystems. Thus, repeating the same reasoning that led to 
Eq. (jSJ), one obtains that the energy distribution law for this subsystem is given by 

= m^BtMi . (10) 

Note, however, that owing to the (intermittent) energy exchange between subsystem n and 
its immediate surrounding, represented by subsystem n — 1, the parameter /3 n is no longer 
constant but rather will fluctuate randomly. If we denote by f(/3 n ) the probability density 
function (PDF) of (3 n , then the marginal distribution p(E) for subsystem n reads 

p(E) = g(E) 1°° m)d/3 n . (11) 



Next we wish to compute /(/3 n ). To do so, we shall make use of Bayesian analysis js]. 
First recall that subsystem n is embedded in a much larger subsystem n — 1, characterized by 
the parameter /3 n _i, which is assumed to vary much slower than (3 n . We are thus interested 
in computing f (fi n \fin-i) ■ From Bayes' theorem 9| one has 

/(fe^il^aKfil^/^^-i), (12) 



where /3 n _i is considered a (hyper)parameter of the distribution of (3 n . In Bayesian parlance, 
the distribution f((3 n ; P n -i) is called the prior distribution, p(E\(3 n ) is the likelihood function, 
and f(fi n ; f3 n _ 1 \E) is the posterior distribution. [In Eq. (TT2|) we introduced the notation 
f(Pn>Pn-i) = f(Pn\Pn-i) for convenience.] 

Let us assume, as is often done in Bayesian analysis, that the prior distribution is con- 
jugate to the likelihood p(E\(3 n ), meaning that the posterior distribution follows the same 
parametric form as the prior distribution. If we consider the rather general case where 
g(E) oc E^~ l , 7 > 0, so that Z (p n ) oc it then follows from Eq. © that p(E\p n ) is 
given by 

p(E\p n ) ex P^E 1 - 1 exp (-P n E) . (13) 
which when viewed as the likelihood of the parameter p n is proportional to a gamma dis- 



tribution. Now, it is well known 
distribution, and so we have 



9| that in this case the conjugate prior is also a gamma 



^-^j^i^rA-jt)' < i4) 

where a is a constant. In obtaining Eq. (|14p we also used the fact that (f3 n \Pn-i) — Pn-i, as 
it should, since subsystem n—1 acts as a heat reservoir for subsystem n. By scale invariance, 
we assume that the distribution f(f3j\f3j-i), for j = 1, n, has the same form as in Eq. (fT4"l) . 
We now have 

POO 

f(Pn) = / f(Pn\Pn-l)f(Pn-l)dP n -l. (15) 

Using this relation recursively then yields 

f(Pn)= ■■■ l[f(P j \P j - 1 )dp 1 .-.dP n - 1 , (16) 

with f(f}j\pj-x) given by Eq. (IT41) . After inserting Eq. (jTBJ) into Eq. (fTTj) . and performing 
a sequence of changes of variables of the type Xj = a(3j/(3j-i, one can show that the re- 
sulting multidimensional integral can be expressed in terms of known higher transcendental 
functions: 

p{E) = 4^ n F (a + 7 + 1, a + 7 + 1; -foa^E), (17) 

where n F (o;i, a n ; —z) is the generalized hypergeometric function of order (n, 0) (l0|. The 

small-scale partition function, Z n , is given by 

" a^T(a + 1) " ~ 
J(a + 7 + l)_ 



Zo(Pa) 



(18) 
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One important property of the generalized hypergeometric (GHG) distribution given in 
(j!7p is that it exhibits power-law tails of the form: p(E) oc E~( a+2 ^ Aor E — > oo. This follows 
immediately from the asymptotic expansion of the function n F n F (ai, ...,a n ] —x) = 
Y2"i=i CiX~ a " (1 + 0(l/x)), as x — > oo. It is also worth pointing out that the first two mem- 
bers of the family n F yield elementary functions, namely, qFq(x) = exp(x) and i-F (l/(g — 
1), x) = exp g (x/(q — 1)), where exp q (x) is the g-exponential: exp q (x) = [1 + (1 — q)x\ l ^ 1 ~ q ' . 
The GHG distribution with n = thus recovers the Boltzmann-Gibbs distribution, whereas 
for n = 1 it g,ves the ,-exponent.a. or T 8 a. lis distribution Q. One the. see 8 f r o,„ the pre- 
ceding discussion that if a system with only one time scale is in thermal equilibrium then the 
Boltzmann-Gibbs distribution follows, whereas if it has two distinct time scales the Tsallis 
distribution should be applicable. For complex systems with more than two characteristic 
,ime scales, such as turbulent flows, GHG distributions of higher order are thus required 
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IV. CONCLUSIONS 



We have presented an alternative derivation, based on Bayesian analysis, of the multi- 
canonical distribution, which describes the statistical equilibrium of complex systems pos- 
sessing a hierarchy of time and length scales. We have shown that the multicanonical dis- 
tribution can be written explicitly in terms of generalized hypergeometric functions, which 
exhibit a power-law asymptotic behavior. This thus shows that the emergence of power law 
distributions — an ubiquitous feature in nature — is intimately connected with the existence 
of multiple time and length scales in the system. 
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